Productive composition of extreme-scale applications using implicitly parallel dataflow
نویسندگان
چکیده
In every decade since the 1970’s, computer scientists have re-examined dataflow-based execution models, hoping the programming productivity benefits these models promise can be realized on practical hardware platforms to implement useful applications. Based on the recent Swift/T implementation of “implicitly parallel functional dataflow” (IPFD) for extremescale systems, we believe that the dataflow model can now provide significant software productivity benefits for developing the upper levels of petascale and exascale applications. We believe dataflow models can also yield significant secondary benefits in verifiability, reliability, and energy savings – areas that are closely related to overall systems development productivity at extreme scale. We discuss here the opportunities for software productivity at these scaled presented by the use of IPFD, and the research and development efforts needed to realize these benefits.
منابع مشابه
ExM: High-level dataflow programming for extreme-scale systems
We present here the ExM (extreme-scale many-task) programming and execution model as a practical solution to the challenges of programing the higher-level logic of complex parallel applications on current petascale and future exascale computing systems. ExM provides an expressive, high-level functional programming model that yields massive concurrency through implicit, automated parallelism. It...
متن کاملImplicitly-Parallel Functional Dataflow for Productive Cloud Programming on Chameleon
One solution that makes parallel programming implicit rather than explicit is the dataflow model. Conceived ~35 years ago, it has only recently been made practical through systems such as Dryad and Swift [1]. We believe that we have successfully created a base for an implicitlyparallel functional dataflow programming model, as exemplified by Swift, a workflow language for executing scientific a...
متن کاملImproving performance of adaptive component-based dataflow middleware
Making the best use of modern computational resources for distributed applications requires expert knowledge of low-level programming tools, or a productive high-level and high-performance programming framework. Unfortunately, even state-of-the-art high-level frameworks still require the developer to conduct a tedious manual tuning step to find the work partitioning which gives the best applica...
متن کاملDataflow Java: Implicitly Parallel Java
Dataflow computation models enable simpler and more efficient management of the memory hierarchy a key barrier to the performance of many parallel programs. This paper describes a dataflow language based on Java. Use of the dataflow model enables a programmer to generate parallel programs without explicit directions for message passing, work allocation and synchronisation. A small handful of ad...
متن کاملPerformance tuning scientific codes for dataflow execution
Performance tuning programs for dataflow execution involves tradeoffs and optimizations which may be significantly different than for execution on conventional machines. We examine some tuning techniques for scientific programs with regular control but irregular geometry. We use as an example the core of an ocean modeling code developed in the implicitly parallel language Id for the Monsoon dat...
متن کامل